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Terminal Disclaimer 

The terminal disclaimer filed on July 2, 2007 disclaiming the terminal portion of 
any patent granted on this application which would extend beyond the expiration date of 
U.S. Patent Number 6,853,398 has been reviewed and is accepted. The terminal 
disclaimer has been recorded. 

Response to Amendment 

Applicant's arguments, see Remarks, filed July 2, 2007 with respect to claims 1- 
38 have been fully considered and are persuasive. The obviousness-type double 
patenting of rejection of claims 1-38 has been withdrawn in response to the Terminal 
Disclaimer obviating the double patenting rejection. 

EXAMINER'S AMENDMENT 

An examiner's amendment to the record appears below. Should the changes 
and/or additions be unacceptable to applicant, an amendment may be filed as provided 
by 37 CFR 1 .312. To ensure consideration of such an amendment, it MUST be 
submitted no later than the payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview 
with Joel Youngs on July 20, 2007. 



In the CLAIMS: 

Cancellation of Claims: Cancel claims 21-28,30-36. 
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REASONS FOR ALLOWANCE 
Claims 1,2,4-15,17-20,37 and 38 are allowed. 

The following is an examiner's statement of reasons for allowance: 
Independent claim 1 discloses a method of real-time rendering in a gaming 
environment to create an interactive experience comprising capturing a plurality of real- 
time video streams of a local participant from a plurality of camera viewpoints; 
generating real-time renderings of said local participant based on said plurality of real- 
time video streams by applying a new view synthesis technique, said real-time 
renderings taken from a perspective of a remote participant located remotely in said 
gaming environment, wherein said new view synthesis technique comprises an imaqe- 
based visual hull technique that comprises approximating a visual hull of said local 
participant by projecting contours associated with said local participant into three- 
dimensional space and computing an intersection of resultinq frusta; and 
sending said real-time renderings to said remote participant for viewing within said 
gaming environment. 

This is patentable over the prior art as the prior art does not disclose the method 
as claimed. Specifically, in the invention disclosed by Eilat et al. (US 6,227,974), a 
single picture of a local player is taken and mapped to a pre-existing avatar for use in a 
three-dimensional gaming environment that may be viewed by a remote player. Thus, 
while an avatar featuring the player's face is utilized, there is no disclosure of a plurality 
of real time video streams wherein a real-time rendering of a local participant is created 
by applying a new view synthesis technique comprising an image-based visual hull of 
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said local participant. There is no suggestion or motivation to modify Eilat et al. such 
that claim 1 of the instant application would be an obvious variation of the invention 
disclosed by Eilat et al., as the new view synthesis technique comprising estimating an 
image-based visual hull requires the simultaneous presence of multiple viewpoints. 

In the invention disclosed by Oh (US 5,616,078), a video game system monitors 
movements made by a player during a gaming session and displays said movements in 
a virtual environment. However, this is accomplished using sensors attached to the 
player's body at various locations, not by capturing a plurality of real-time video streams 
from a plurality of camera viewpoints and applying a new view synthesis technique 
comprising an image based visual hull. 

In the invention disclosed by Kage (US 6,335,977 B1), multiple cameras are 
utilized to monitor the movements of a participant and reproduce said movements using 
computer graphics. However, Kage does not disclose utilizing an image-based visual 
hull technique that comprises approximating a visual hull of said participant by 
projecting contours associated with said local participant into three-dimensional space 
and computing an intersection of resulting frusta, nor does Kage disclose generating 
real-time renderings taken from a perspective of a remote participant and sending said 
real-time renderings to said remote participant for viewing within a gaming environment. 

In the disclosure of "The Visual Hull Concept for Silhouette-Based Image 
Understanding" by Aldo Laurentini (IEEE Transactions on Pattern Analysis and Machine 
Intelligence 1994), herein referred to as Laurentini, Laurentini discloses an image-based 
visual hull technique that comprises approximating a visual hull of an object by 
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projecting contours associated with said object into three-dimensional space and 
computing an intersection of resulting frusta. However, Laurentini provides no 
disclosure of generating real-time renderings of a participating game player based on a 
plurality of real-time video streams, wherein said real-time renderings are taken from a 
perspective of a remote participant located remotely in a gaming environment. 

Thus, while it is well known in the art to apply a still image of a player to a 
character in a gaming environment in order to make said character resemble said 
player, there appears to be no teaching of the limitations of the method as disclosed in 
claim 1. 

Claim 1 1 contains limitations similar to those of claim 1 , and is therefore also 
allowable over the prior art. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

Pertinent Prior Art 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Pertinent prior art includes: 

• Eilat et al. (US 6,227,974 B1), drawn to an interactive gaming system wherein a 
picture of the player is incorporated into a virtual gaming environment. 
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• Breslow et al. (US 4,710,873), drawn to a video game incorporating digitized 
images of a player into game graphics. 

• Oh (US 5,616,078), drawn to a video game system wherein a player's 
movements are tracked and displayed utilizing motion detecting sensors. 

• Yoshigahara et al. (US 6,476,812 B1), drawn to a method for detecting a collision 
between a space and'an object comprising a plurality of cameras and a process 
determining section for creating a three-dimensional image of an object colliding 
against a virtual space. 

• Cok (US 6,980,333 B2), drawn to a motion imaging system for generating a real- 
time motion image sequence of an environment and integrating a motion image 
sequence of the viewer into the computer generated motion image sequence of 
the environment to produce a composite image. 

• Oishi et al. (US 6,890,262 B2), drawn to a video game apparatus for controlling 
viewpoint movement of a simulated camera in a video game. 

• Ohshima et al. (US 6,522,312 B2), drawn to an apparatus for presenting mixed 
reality shared among operators. 

• Berg et al. (US 7,048,630 B2), drawn to a system and method for capturing a 
player's image for incorporation into a game. 

• Marks (US 6,795,068 B1), drawn to a prop input device and method for mapping 
an object from a two-dimensional camera image to a three-dimensional space for 
controlling action in a game. 
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• Smooth et al. (US 2004/0102247 A1), drawn to a surface-based interactive 
environment utilizing an infrared light source for capturing participant motion. 

• Kanarat (US 2003/0130035 A1), drawn to an amusement machine for capturing 
a participant image and comparing said image to a pre-stored image. 

• Yamade et al. (US 6,970,177 B2), drawn to an image processing system for 
capturing a player image and incorporating said image into a game. 

• Swamo et al. (US 6,677,967 B2), drawn to a video game system for capturing 
images and applying the captured images to animated game play characters. 

• Erdem (US 6,664,956 B1), drawn to a method for generating a personalized 3-D 
face model. 

• Liu et al. (US 6,807,290 B2), drawn to a rapid computer modeling method of 
faces for animation. 

• Marschner et al. (US 7,098,920 B2), drawn to a method and system for 
animating facial features and expression transformation. 

• Waupotitsch et al. (US 7,016,824 B2), drawn to an interactive try-on platform 
wherein a user's three-dimensional image is generated from a series of still 
images. 

• Fujiwara et al. (US 6,977,660 B2), drawn to a data processing apparatus for a 
three-dimensional image processing technique of generating a texture image 
from an original three-dimensional model. 

• Kang (US 6,009,210), drawn to a hands-free interface to a virtual reality 
environment using head tracking. 
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• Erdem (US 7,127,081 B1), drawn to a method for tracking motion of a face and 
generating a three-dimensional model of said face. 

• Paul et al. (US 7,121,946 B2), drawn to a real-time head tracking system for 
computer games and other applications. 

• Kage (US 6,335,977 B1), drawn to an action recognizing apparatus and 
recording medium. 

• Malzbender (US7,057,662), drawn to an apparatus featuring a plurality of 
cameras. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Meagan Thomasson whose telephone number is (571) 
272-2080. The examiner can normally be reached on M-F 830-5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Xuan Thai can be reached on (571) 272-7147. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Meagan Thomasson 
July 20, 2007 
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